A confirmatory factorial analysis of the Chatbot Usability Scale: a multilanguage validation

Abstract The Bot Usability Scale (BUS) is a standardised tool to assess and compare the satisfaction of users after interacting with chatbots support development usable conversational systems. English version 15-item BUS scale (BUS-15) was result an exploratory factorial analysis; confirmatory analysis tests replicability initial model further explores properties aiming optimise this seeking for stability original model, potential reduction items, testing multiple language versions scale. BUS-15 usability metrics user experience (UMUX-LITE), used here convergent validity purposes, were translated from Spanish, German, Dutch. A total 1292 questionnaires completed in languages; these collected 209 participants overall pool 26 chatbots. acceptably reliable; however, shorter more reliable solution 11 items (BUS-11) emerged data. ratings obtained BUS-11 not significantly different English, suggesting that could be languages. results also suggested age seems affect evaluation when using scale, older rating as less satisfactory, compared younger participants. In line expectations, based on reliability, positively correlates UMUX-LITE new aims facilitate chatbots, its diffusion help practitioners performances benchmark during product assessment stage. This way harmonise enable comparability field human agent interaction.